{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 作业三：\n",
    "写一个可以将MNIST图片向任意方向（上，下，左，右）移动一个像素功能。然后对训练集中的每张图片，创建四个位移后的副本，每个方向一个，添加到训练集。最后，在这个扩展过的训练集上训练模型，衡量其在测试集上的精度，来优化精度，这种人工扩展训练集的技术成为数据增广或训练集扩展 15分"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "c:\\users\\yxx\\appdata\\local\\programs\\python\\python38\\lib\\site-packages\\sklearn\\feature_extraction\\text.py:17: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated since Python 3.3, and in 3.9 it will stop working\n",
      "  from collections import Mapping, defaultdict\n"
     ]
    }
   ],
   "source": [
    "from sklearn.datasets import fetch_mldata\n",
    "import numpy as np\n",
    "import pandas as pd \n",
    "import matplotlib.pyplot as plt"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 数据集读取\n",
    "mnist = fetch_mldata('mnist-original', data_home='./')\n",
    "X=mnist[\"data\"]\n",
    "y=mnist[\"target\"]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "#打乱顺序\n",
    "shuffle_index=np.random.permutation(70000)\n",
    "X=X[shuffle_index]\n",
    "y=y[shuffle_index]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 拆分训练集与测试集\n",
    "X_train,y_train,X_test,y_tes=X[:60000],y[:60000],X[60000:],y[60000:]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "import matplotlib"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAPsAAAD4CAYAAAAq5pAIAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy86wFpkAAAACXBIWXMAAAsTAAALEwEAmpwYAAANyElEQVR4nO3db6xU9Z3H8c9nXYzGisByRWJxacUHazaurTdEwqps6jaKJsiDbqqmaqKLUUxa0sRF1qTGBwY3S01jFpPbRYsbtDYpRBN1KTFNTH2gXAmLsGQFDFaQcC8alD5QF/zug3vcXPHOby5z5h9+369kMjPnO2fOlxM+98zMb878HBEC8PX3Z71uAEB3EHYgCcIOJEHYgSQIO5DEn3dzYzNnzoy5c+d2c5NAKvv379eRI0c8Ua1W2G1fJ+kXks6Q9O8Rsbr0+Llz52p4eLjOJgEUDA4ONqy1/DLe9hmS/k3S9ZIulXSz7UtbfT4AnVXnPft8SXsj4p2I+EzSryUtaU9bANqtTtgvlPTeuPsHqmVfYnuZ7WHbw6OjozU2B6COOmGf6EOAr3z3NiKGImIwIgYHBgZqbA5AHXXCfkDSnHH3vynp/XrtAOiUOmHfKukS29+yfaakH0p6oT1tAWi3lofeIuK47fskbdbY0NuTEbGrbZ0BaKta4+wR8ZKkl9rUC4AO4uuyQBKEHUiCsANJEHYgCcIOJEHYgSQIO5AEYQeSIOxAEoQdSIKwA0kQdiAJwg4kQdiBJAg7kARhB5Ig7EAShB1IgrADSRB2IAnCDiRB2IEkCDuQBGEHkiDsQBKEHUiCsANJEHYgCcIOJFFrFlfgdLVv375i/ZFHHqn1/DfeeGOxvnTp0lrP34paYbe9X9IxSSckHY+IwXY0BaD92nFk/7uIONKG5wHQQbxnB5KoG/aQ9Dvbb9peNtEDbC+zPWx7eHR0tObmALSqbtgXRsR3JV0vabntq09+QEQMRcRgRAwODAzU3ByAVtUKe0S8X12PSNokaX47mgLQfi2H3fY5ts/94rak70va2a7GALRXnU/jZ0naZPuL53kmIv6zLV0hhU8//bRYP3bsWLH+4osvFus7duxoWNuwYUNx3ZGRkWK9mW3bthXrp9U4e0S8I+lv2tgLgA5i6A1IgrADSRB2IAnCDiRB2IEkOMUVtXzwwQfF+hNPPNGw1mzo7PXXX2+ppy9ERMNaNWTcMb0YWmuGIzuQBGEHkiDsQBKEHUiCsANJEHYgCcIOJME4e3J79uwp1tesWVOsP/fcc8X6Rx99dMo9Tda0adOK9csuu6xhbcWKFcV1L7roolr1qVOnFuu9wJEdSIKwA0kQdiAJwg4kQdiBJAg7kARhB5JgnP1roPSTyatXry6uu3HjxmL9s88+K9ZL54xL0tlnn92wduuttxbXveWWW4r1OXPmFOvz5s0r1rPhyA4kQdiBJAg7kARhB5Ig7EAShB1IgrADSTDOfhpods75Nddc07B29OjRWtu+4IILivU77rijWF+5cmXD2nnnnddKS2hR0yO77Sdtj9jeOW7ZDNtbbO+prqd3tk0AdU3mZfyvJF130rKVkl6JiEskvVLdB9DHmoY9Il6V9OFJi5dIWl/dXi/ppva2BaDdWv2AblZEHJKk6vr8Rg+0vcz2sO3h0dHRFjcHoK6OfxofEUMRMRgRgwMDA53eHIAGWg37YduzJam6HmlfSwA6odWwvyDp9ur27ZKeb087ADql6Ti77WclLZI00/YBST+TtFrSb2zfKemPkn7QySazW758ebFe+m32ZvOQP/3008X6tddeW6w3G4dH/2ga9oi4uUHpe23uBUAH8XVZIAnCDiRB2IEkCDuQBGEHkuAU19NAs1Nc69i7d2+xvnjx4o5tG93FkR1IgrADSRB2IAnCDiRB2IEkCDuQBGEHkmCc/TTwzDPPFOtLly5tWBsZKf+uyMMPP1ys79u3r1h//PHHi/Vp06YV6+gejuxAEoQdSIKwA0kQdiAJwg4kQdiBJAg7kATj7KeBBQsWFOuvvfZaw9q9995bXHfLli3F+oYNG4r1KVOmFOuPPfZYwxpTNncXR3YgCcIOJEHYgSQIO5AEYQeSIOxAEoQdSIJx9q+Biy++uGFt8+bNxXXXrl1brK9atapYf+qpp4r1Xbt2Nay9/PLLxXVnzJhRrOPUND2y237S9ojtneOWPWT7oO3t1YWZBIA+N5mX8b+SdN0Eyx+LiMury0vtbQtAuzUNe0S8KunDLvQCoIPqfEB3n+0d1cv86Y0eZHuZ7WHbw6OjozU2B6COVsP+hKSLJV0u6ZCkNY0eGBFDETEYEYMDAwMtbg5AXS2FPSIOR8SJiPhc0i8lzW9vWwDaraWw25497u5SSTsbPRZAf2g6zm77WUmLJM20fUDSzyQtsn25pJC0X9LdnWsRndTsfHfbtdbfunVrw1qzMf4HH3ywWMepaRr2iLh5gsXrOtALgA7i67JAEoQdSIKwA0kQdiAJwg4kwSmuKLrnnnuK9dIprJK0bl3jgZuDBw8W1/3kk0+K9bPOOqtYx5dxZAeSIOxAEoQdSIKwA0kQdiAJwg4kQdiBJBhnR9Hbb79drB8+fLhYP378eMPaG2+8UVz3xIkTxTpODUd2IAnCDiRB2IEkCDuQBGEHkiDsQBKEHUiCcfbkjh49WqyvXLmyWN+0aVOxXvop6nPPPbe47plnnlms49RwZAeSIOxAEoQdSIKwA0kQdiAJwg4kQdiBJBhn/5rbvHlzsX7//fcX6zt27Ki1/dJYerMpmadMmVJr2/iypkd223Ns/972btu7bP+4Wj7D9hbbe6rr6Z1vF0CrJvMy/rikn0bEX0m6UtJy25dKWinplYi4RNIr1X0Afapp2CPiUERsq24fk7Rb0oWSlkhaXz1svaSbOtQjgDY4pQ/obM+V9B1Jr0uaFRGHpLE/CJLOb7DOMtvDtodHR0drtgugVZMOu+1vSPqtpJ9ExMeTXS8ihiJiMCIGBwYGWukRQBtMKuy2p2gs6BsiYmO1+LDt2VV9tqSRzrQIoB2aDr157BzFdZJ2R8TPx5VekHS7pNXV9fMd6bBLjh07VqyXTuW87bbbius2m3q4Wf3AgQPF+l133dWwtnv37uK6zf7dpVNUJWnRokXF+tDQUMPavHnziuuivSYzzr5Q0o8kvWV7e7VslcZC/hvbd0r6o6QfdKRDAG3RNOwR8QdJjf68f6+97QDoFL4uCyRB2IEkCDuQBGEHkiDsQBKc4lp54IEHivV169Y1rDX7OeV33323WH/vvfeK9SNHjhTrzcbCS6ZOnVqs33333cX6o48+2vK20V0c2YEkCDuQBGEHkiDsQBKEHUiCsANJEHYgCcbZKzfccEOxvnbt2oa155/v7Kn8s2fPLtavvvrqhrUrrriiuO6VV15ZrF911VXFOk4fHNmBJAg7kARhB5Ig7EAShB1IgrADSRB2IAnG2SsLFy4s1lesWNGw9vHHk54gZ0LNxvgXLFhQrM+aNavW9pEDR3YgCcIOJEHYgSQIO5AEYQeSIOxAEoQdSGIy87PPkfS0pAskfS5pKCJ+YfshSf8oabR66KqIeKlTjXZas99PX7NmTZc6ATpjMl+qOS7ppxGxzfa5kt60vaWqPRYR/9q59gC0y2TmZz8k6VB1+5jt3ZIu7HRjANrrlN6z254r6TuSXq8W3Wd7h+0nbU9vsM4y28O2h0dHRyd6CIAumHTYbX9D0m8l/SQiPpb0hKSLJV2usSP/hG9qI2IoIgYjYnBgYKB+xwBaMqmw256isaBviIiNkhQRhyPiRER8LumXkuZ3rk0AdTUNu8emCF0naXdE/Hzc8vE/ebpU0s72twegXSbzafxCST+S9Jbt7dWyVZJutn25pJC0X1J5bl8APTWZT+P/IGmiCcBP2zF1ICO+QQckQdiBJAg7kARhB5Ig7EAShB1IgrADSRB2IAnCDiRB2IEkCDuQBGEHkiDsQBKEHUjCEdG9jdmjkt4dt2impCNda+DU9Gtv/dqXRG+tamdvfxkRE/7+W1fD/pWN28MRMdizBgr6tbd+7Uuit1Z1qzdexgNJEHYgiV6HfajH2y/p1976tS+J3lrVld56+p4dQPf0+sgOoEsIO5BET8Ju+zrb/2N7r+2VveihEdv7bb9le7vt4R738qTtEds7xy2bYXuL7T3V9YRz7PWot4dsH6z23Xbbi3vU2xzbv7e92/Yu2z+ulvd03xX66sp+6/p7dttnSHpb0t9LOiBpq6SbI+K/u9pIA7b3SxqMiJ5/AcP21ZL+JOnpiPjratm/SPowIlZXfyinR8Q/9UlvD0n6U6+n8a5mK5o9fppxSTdJukM93HeFvv5BXdhvvTiyz5e0NyLeiYjPJP1a0pIe9NH3IuJVSR+etHiJpPXV7fUa+8/SdQ166wsRcSgitlW3j0n6Yprxnu67Ql9d0YuwXyjpvXH3D6i/5nsPSb+z/abtZb1uZgKzIuKQNPafR9L5Pe7nZE2n8e6mk6YZ75t918r053X1IuwTTSXVT+N/CyPiu5Kul7S8ermKyZnUNN7dMsE0432h1enP6+pF2A9ImjPu/jclvd+DPiYUEe9X1yOSNqn/pqI+/MUMutX1SI/7+X/9NI33RNOMqw/2XS+nP+9F2LdKusT2t2yfKemHkl7oQR9fYfuc6oMT2T5H0vfVf1NRvyDp9ur27ZKe72EvX9Iv03g3mmZcPd53PZ/+PCK6fpG0WGOfyO+T9M+96KFBX9+W9F/VZVeve5P0rMZe1v2vxl4R3SnpLyS9ImlPdT2jj3r7D0lvSdqhsWDN7lFvf6uxt4Y7JG2vLot7ve8KfXVlv/F1WSAJvkEHJEHYgSQIO5AEYQeSIOxAEoQdSIKwA0n8HxaZKeq+UWDVAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "#展示图片\n",
    "%matplotlib inline\n",
    "some_digit=X_train[40000]\n",
    "some_digit_img=some_digit.reshape(28,28)\n",
    "plt.imshow(some_digit_img,cmap=matplotlib.cm.binary)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [],
   "source": [
    "def image_offset(data_s,dir='u',offset=1):\n",
    "    data_r=data_s.reshape((len(data_s), int(np.sqrt(data_s.shape[1])), -1))\n",
    "    data_new=[]\n",
    "    if dir=='u':\n",
    "        for img_data in data_r:\n",
    "            img_data_new=np.concatenate((img_data[offset:,:],img_data[:offset,:]),axis=0)\n",
    "            data_new.append(img_data_new)\n",
    "        \n",
    "    elif dir=='d':\n",
    "        for img_data in data_r:\n",
    "            img_data_new=np.concatenate((img_data[(len(img_data)-offset):,:],img_data[:(len(img_data)-offset),:]),axis=0)\n",
    "            data_new.append(img_data_new)\n",
    "            \n",
    "    elif dir=='l':\n",
    "        for img_data in data_r:\n",
    "            img_data_new=np.concatenate((img_data[:,offset:],img_data[:,:offset]),axis=1)\n",
    "            data_new.append(img_data_new)\n",
    "            \n",
    "    elif dir=='r':\n",
    "        for img_data in data_r:\n",
    "            img_data_new=np.concatenate((img_data[:,(len(img_data)-offset):],img_data[:,:(len(img_data)-offset)]),axis=1)\n",
    "            data_new.append(img_data_new)\n",
    "    data_new=np.array(data_new)        \n",
    "    X_train=data_new.reshape(len(data_s),-1)\n",
    "    return X_train"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [],
   "source": [
    "X_train_u=image_offset(X_train,dir=\"u\")\n",
    "X_train_d=image_offset(X_train,dir='d')\n",
    "X_train_l=image_offset(X_train,dir='l')\n",
    "X_train_r=image_offset(X_train,dir='r')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [],
   "source": [
    "X_train_new=np.concatenate((X_train,X_train_u,X_train_d,X_train_l,X_train_r),axis=0)\n",
    "y_train_new=np.concatenate((y_train,y_train,y_train,y_train,y_train),axis=0)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [],
   "source": [
    "#KN 分类器\n",
    "from sklearn.neighbors import KNeighborsClassifier\n",
    "kn_clf=KNeighborsClassifier()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "KNeighborsClassifier(algorithm='auto', leaf_size=30, metric='minkowski',\n",
       "           metric_params=None, n_jobs=1, n_neighbors=5, p=2,\n",
       "           weights='uniform')"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "kn_clf.fit(X_train,y_train)#填充数据"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([5.])"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#验证\n",
    "kn_clf.predict([some_digit])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.metrics import precision_score,recall_score\n",
    "from sklearn.model_selection import cross_val_score\n",
    "from sklearn.model_selection import cross_val_predict"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([False, False, False, ..., False, False, False])"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "y_train_5=(y_train.astype(int)==5)\n",
    "y_train_5"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "KNeighborsClassifier(algorithm='auto', leaf_size=30, metric='minkowski',\n",
       "           metric_params=None, n_jobs=1, n_neighbors=5, p=2,\n",
       "           weights='uniform')"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "kn_clf.fit(X_train,y_train_5)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "y_train_pred=cross_val_predict(kn_clf,X_train,y_train_5,cv=3)#预测的结果"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 计算精度与召回\n",
    "precision=precision_score(y_train_5,y_train_pred)\n",
    "recall=recall_score(y_train_5,y_train_pred)\n",
    "print('precision:{:.1f}%'.format(precision*100))\n",
    "print('recall:{:.1f}%'.format(recall*100))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 对比新的训练集的效果"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "y_train_new_5=(y_train_new.astype(int)==5)\n",
    "kn_clf_new.fit(X_train_new,y_train_new_5)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "kn_clf_new.predict([some_digit])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "y_train_new_pred=cross_val_predict(kn_clf_new,X_train_new,y_train_new_5,cv=3)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#新的训练集上计算精度与召回\n",
    "precision=precision_score(y_train_new_5,y_train_new_pred)\n",
    "recall=recall_score(y_train_new_5,y_train_new_pred)\n",
    "print('precision:{:.1f}%'.format(precision*100))\n",
    "print('recall:{:.1f}%'.format(recall*100))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#计算在测试集上的效果\n",
    "y_test_pred=cross_val_predict(kn_clf_new,X_test,y_test_5,cv=3)\n",
    "precision=precision_score(y_test_5,y_test_pred)\n",
    "recall=recall_score(y_test_5,y_test_pred)\n",
    "print('precision:{:.1f}%'.format(precision*100))\n",
    "print('recall:{:.1f}%'.format(recall*100))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 结论\n",
    "- 没有进行数据增广的表现：\n",
    "    - precision:97.0%\n",
    "    - recall:95.5%\n",
    "- 像素位移后，模型表现\n",
    "    - precision:98.0%\n",
    "    - recall:97.2%\n",
    "- 测试集表现\n",
    "    - precision:95.3%\n",
    "    - recall:91.1%"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}
